Simultaneous test construction by zero-one programming

A method is described for simultaneous test construction using the Operations Research technique zero-one programming. The model for zero-one programming consists of two parts. The first contains the objective function that describes the aspect to be optimized. The second part contains the constraints under which the objective function should be optimized. The selection of items is based on information from item response theory. Simultaneous test design is used when tests have to be constructed so that there is a certain relationship between them. Two examples of simultaneous test construction are presented. The construction of two parallel tests is considered, and designs of three tests that should measure best at successive parts of the ability scale are described. Examples were carried out using an item bank of 10 items chosen at random. The sequential construction of the same series of tests was compared. The examples illustrate that the tests constructed using simultaneous techniques best fit the intentions of the test constructor

The construction of parallel tests from IRT-based item banks

The construction of parallel tests from item response theory (IRT) based item banks is discussed. Tests are considered parallel whenever their information functions are identical. After the methods for constructing parallel tests are considered, the computational complexity of 0-1 linear programming and the heuristic procedure applied are discussed. Two methods for selecting parallel tests in succession (sequential test construction) are formulated. The first uses a non-partitioned item bank (Method 1), and the second uses a partitioned item bank (Method 2). Two methods are also reviewed for simultaneous test construction, one for non-partitioned item banks (Method 3) and one for partitioned item banks (Method 4). A heuristic procedure is used for solving these 0-1 linear programming problems. A simulation study compared these methods using two item banks, each consisting of 100 items. Satisfactory results were obtained, both in terms of the amount of central processing unit time needed and the differences between the information functions of the parallel tests selected. It was concluded that when the Rasch model fits the items, sequential test construction methods are preferable. For the three-parameter model, the use of Method 1 is inappropriate

A zero-one programming approach to Gulliksen's matched random subtests method

Gulliksen’s matched random subtests method is a graphical method to split a test into parallel test halves. The method has practical relevance because it maximizes coefficient α as a lower bound to the classical test reliability coefficient. In this paper the same problem is formulated as a zero-one programming problem, the advantage being that it can be solved by computer algorithms that already exist. It is shown how the procedure can be generalized to split tests of any length. The paper concludes with an empirical example comparing Gulliksen’s original hand-method with the zero-one programming version. Index terms: Classical test theory, Gulliksen’s matched random subtests method, Item matching, Linear programming, Parallel tests, Test reliability, Zero-one programming

Achievement test construction using 0-1 linear programming

In educational testing the work of professional test agencies has shown a trend towards item banking. Achievement test construction is viewed as selecting items from a test item bank such that certain specifications are met. As the number of possible tests is large and practice usually imposes various constraints on the selection process, a mathematical programming approach is obvious. In this paper it is shown how to formulate achievement test construction as a 0¿1 linear programming problem. A heuristic for solving the problem is proposed and two examples are given. It is concluded that a 0¿1 linear programming approach fits the problem of test construction in an appropriate way and offers test agencies the possibility of computerizing their services

A zero-one programming approach to Gulliksen's matched random subtests method

In order to estimate the classical coefficient of test reliability, parallel measurements are needed. H. Gulliksen's matched random subtests method, which is a graphical method for splitting a test into parallel test halves, has practical relevance because it maximizes the alpha coefficient as a lower bound of the classical test reliability coefficient. This paper formulates this same problem as a zero-one programming problem, the advantage being that it can be solved by algorithms already existing in computer code. Focus is on giving Gulliksen's method a sound computational basis. How the procedure can be generalized to test splits into components of any length is shown. An empirical illustration of the procedure is provided, which involves the use of the algorithm developed by A. H. Land and A. Doig (1960), as implemented in the LANDO program. Item difficulties and item-test correlations were estimated from a sample of 5,418 subjects--a sample size that is large enough to prevent capitalizing on chance in the Gulliksen method. Two data tables and one graph are provided

IRT-based test construction

Four discussions of test construction based on item response theory (IRT) are presented. The first discussion, "Test Design as Model Building in Mathematical Programming" (T.J.J.M. Theunissen), presents test design as a decision process under certainty. A natural way of modeling this process leads to mathematical programming. General models of test construction are discussed, with information about algorithms and heuristics; ideas about the analysis and refinement of test constraints are also considered. The second paper, "Methods for Simultaneous Test Construction" (Ellen Boekkooi-Timminga), gives an overview of simultaneous test construction using zero-one programming. The item selection process is based on IRT. Some objective functions and practical constraints are presented, the construction of parallel tests is considered, and two tables are provided. The third paper, "Automated Test Construction Using Minimax Programming" (Wim J. van der Linden), proposes the use of the minimax principle in IRT test construction and indicates how this results in test information functions deviating less systematically from the target function than for the usual criterion of minimal test length. An alternative approach and some practical constraints are considered. The final paper, "A Procedure To Assess Target Information Functions" (Henk Kelderman), discusses the concept of an information function and its properties. An interpretable function of information is chosen: the probability of a wrong order of the ability estimates of two subjects

A cluster-based method for test construction

Several methods for optimal test construction from item banks have recently been proposed using information functions. The main problem with these methods is the large amount of time required to identify an optimal test. In this paper, a new method is presented for the Rasch model that considers groups of interchangeable items, instead of individual items. The process of item clustering is described, the cluster-based test construction model is outlined, and the computational procedure and results are given. Results indicate that this method produces accurate results in small amounts of time. Index terms: information functions, item banking, item response theory, linear programming, test construction

A method for designing IRT-based item banks

Since 1985 several procedures for computerized test construction using linear programming techniques have been described in the literature. To apply these procedures successfully, suitable item banks are needed. The problem of designing item banks based on item response theory (IRT) is addressed. A procedure is presented that determines whether an existing item bank meets the test construction requirements. If not, the method indicates which items have to be added to the banks so that it will meet the requirements. The comparison of desired and present item bank characteristics, writing, and calibrating items continues until the characteristics of the item bank are acceptable. Four categories of characteristics are: (1) general characteristics (such as format); (2) subject matter characteristics (such as learning objective); (3) psychometric characteristics (such as IRT-parameters); and (4) user statistics. One figure illustrates the procedure